Identification and analysis of movement using sensor devices

ABSTRACT

Disclosed are various embodiments for using sensor devices to detect and monitor movement of a body. The sensor devices may be coupled to a body in a predefined arrangement, where a sensor device is positioned on a particular portion of the body in accordance with the predefined arrangement. The sensor devices measure a position of the portion of the body to which the band is secured during movement and communicate the position to a client device to be used in generating and updating a graphical representation of the movement performed in near-real-time. Further, the client device may determine whether the movement performed conforms to an ideal movement and, in response to the movement not conforming to the predefined movement, a suggested change in the movement can be identified that, if performed, would conform to the ideal movement.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of and priority to co-pending U.S.patent application Ser. No. 14/881,499, entitled “IDENTIFICATION ANDANALYSIS OF MOVEMENT USING SENSOR DEVICES,” filed Oct. 13, 2015, whichin turn claims the benefit of and priority to U.S. Provisional PatentApplication No. 62/216,487 entitled “IDENTIFICATION AND ANALYSIS OFMOVEMENT USING SENSOR DEVICES,” filed on Sep. 10, 2015, both of whichare incorporated herein by reference in their entirety.

BACKGROUND

Positions and orientations of fixed, rigid bodies, such as a kinematicchain of a robotic manipulator, are tracked using Denavit-Hartenbergconvention and other similar methods. However, detecting position andorientation of various parts of a human remain problematic as size andmakeup of one body may vary from another. Computer vision is commonlyemployed to detect movement of person's body. However, computer visionrequires an imaging device, such as a camera. Additionally, computervision requires a system capable of detecting visual identifiers thatmight not always be accurately detected.

There are various reasons for needing to understand and analyze themovement of a person. In the medical field, screening movement patternsto identify limitations, examine range of motion after an injury, ordocument historical progression is often needed for diagnosis, aid intreatment, and/or rehabilitation. Traditionally, mobility, stability,and coordination of a patient are assessed by doctors, physicaltherapists, and/or other qualified healthcare professionals at a medicalfacility. However, there is a great need to care for individuals inremote areas, where access to specialists can be difficult because ofphysical, economic, and geographic boundaries. Where distance is not abarrier, often age or disability may make it difficult for an individualto travel to a medical facility for diagnosis or treatment. To providethe best outcome for the patient, improved techniques are sought forboth a clinical environment and remote extension.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the present disclosure can be better understood withreference to the following drawings. The components in the drawings arenot necessarily to scale, with emphasis instead being placed uponclearly illustrating the principles of the disclosure. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views.

FIG. 1 is a drawing of a networked environment that includes sensordevices positioned on a body, a client device, and a computingenvironment according to various embodiments of the present disclosure.

FIG. 2 is a drawing of a body performing a movement where sensor data isused to generate a three-dimensional reconstruction of the movement inthe client device of FIG. 1 according to various embodiments of thepresent disclosure.

FIG. 3 is a drawing of another client device that may be employed toshow a three-dimensional reconstruction of the movement of FIG. 2according to various embodiments of the present disclosure.

FIG. 4 is a drawing of a body performing a movement where sensor data iscollected to generate a three-dimensional reconstruction of the movementin a client device according to various embodiments of the presentdisclosure.

FIG. 5 is a drawing of a client device with two three-dimensionalreconstructions that show the movement of FIG. 4 in view of a predefinedmovement according to various embodiments of the present disclosure.

FIGS. 6-7 are drawings of a client device with two three-dimensionalreconstructions that show a movement performed or being performed inview of another predefined movement according to various embodiments ofthe present disclosure.

FIG. 8 is a flowchart illustrating one example of functionalityimplemented as portions of the sensor devices of FIG. 1 according tovarious embodiments of the present disclosure.

FIG. 9 is an example of pseudocode for reading sensor data according tovarious embodiments of the present disclosure.

FIG. 10 is a flowchart illustrating one example of functionalityimplemented as portions of the client device of FIG. 1 according tovarious embodiments of the present disclosure.

FIG. 11 is an example of pseudocode for processing a sketch according tovarious embodiments of the present disclosure

FIG. 12 is a flowchart illustrating one example of functionalityimplemented as portions of the computing environment of FIG. 1 accordingto various embodiments of the present disclosure.

FIG. 13 is a schematic block diagram that provides one exampleillustration of a sensor device of employed in the networked environmentof FIG. 1 according to various embodiments of the present disclosure.

FIG. 14 is a schematic block diagram that provides one exampleillustration of a client device of employed in the networked environmentof FIG. 1 according to various embodiments of the present disclosure.

FIG. 15 is a schematic block diagram that provides one exampleillustration of a computing environment employed in the networkedenvironment of FIG. 1 according to various embodiments of the presentdisclosure.

DETAILED DESCRIPTION

The present disclosure relates to using sensors to identify movements,such as body movements performed by a human, animal, humanoid, animoid,or other object capable of performing various types of movements. Asnoted above, there are various reasons for needing to understand andanalyze the movement of a person. In the medical field, screeningmovement patterns to identify limitations, examine range of motion afteran injury, or document historical progression is often needed fordiagnosis, aid in treatment, and/or rehabilitation. Traditionally,mobility, stability, and coordination of a patient are assessed bydoctors, physical therapists, and/or other qualified healthcareprofessionals at a medical facility. However, there is a great need tocare for individuals in remote areas, where access to a specialist canbe difficult because of physical, economic, and geographic boundaries.Where distance is not a barrier, often age or disability may make itdifficult for an individual to travel to a medical facility fordiagnosis or treatment. To provide the best outcome for the patient,improved techniques are sought for both a clinical environment andremote extension.

Previous technological applications have analyzed movement using digitalimage processing and computer vision. For example, users may be requiredto position markers or other visual identifiers on their body, and acamera and computer system tracks the movement when the markers or othervisual identifiers are detected and the position has changed from afirst frame to a second frame. However, these systems are limited incapability as they are limited to current technologies in digital imageprocessing, require a camera, and the accuracy in detecting bodymovement is not ideal. For example, it is difficult to identify smallmovements in various joints, such as the knee.

Accordingly, it is beneficial for a system to be capable of monitoringmovement without having to utilize visual analysis. According to variousembodiments of the present disclosure, various sensor devices can bepositioned on a body of a person, animal, humanoid, animoid, or othermoveable body to accurately track movement as will be described. Eachsensor device may be coupled to a band or other attachment that securesthe sensor device to a position on the body. Additionally, each sensordevice may include circuitry that measures a position of the portion ofthe body to which the band is secured during a movement performed by theportion of the body. In various embodiments, each sensor device maycomprise a microcontroller, an accelerometer, a gyroscope, acommunication module, and/or other components as may be appreciated.

In one embodiment, at least one of the sensor devices may be dedicatedas a “master” sensor device, where other ones of the sensor devicescomprise “slave” sensor devices. The slave sensor devices may collectsensor data and communicate the sensor data to the master sensor device.The master sensor device may communicate the sensor data, as well assensor data collected by the master sensor device itself, to a computingdevice for processing. In some embodiments, individual sensor devicescan independently communicate data to the client device and/or thecomputing environment. In various embodiments, the computing devicecomprises a client device, such as a smartphone, a tablet computingdevice, a laptop computing device, a smartwatch, or other computingdevice as will be described.

Further, in various embodiments, the sensor devices may be placed inaccordance with a predefined arrangement stored in memory, such that acomputing device can correlate each of the sensor devices to arespective position of a body. In one embodiment, assuming the body isthat of a person, a first sensor device may be positioned at a firstshoulder of the body; a second sensor device may be positioned at asecond shoulder of the body; a third sensor device may be positioned ata first elbow of the body; a fourth sensor device may be positioned at asecond elbow of the body; a fifth sensor device may be positioned at afirst wrist of the body; a sixth sensor device may be positioned at asecond wrist of the body; a seventh sensor device may be positioned at afirst knee of the body; an eighth sensor device may be positioned at asecond knee of the body; a ninth sensor device may be positioned at afirst ankle of the body; a tenth sensor device may be positioned at asecond ankle of the body; an eleventh sensor device may be positioned ata first side of hips of the body; and a twelfth sensor device may bepositioned at a second side of hips of the body. As may be appreciated,in alternative embodiments, a lesser amount of sensor devices may beutilized to accurately estimate a position of a portion of a body. Forexample, position estimation may be improved using a lesser amount ofsensor devices based on correlations made between different portions ofbody parts (e.g., shoulder position based on position of elbow). To thisend, other arrangements of sensor devices may be utilized, asapplicable. In some embodiments, additional sensors may be located atthe forearm, hand, thigh, foot, neck, and/or head to further markmovement. In other embodiments, sensor devices can be attached to anobject coupled to the body, such as an artificial limb or otherprosthetic; sports equipment, such as a golf club, vaulting pole,baseball bat, etc.; or an object that may decouple from the body, suchas a baseball.

As noted above, the sensor devices may include a communication module,such as a Bluetooth® module, a Bluetooth® low-energy module, a ZigBee®module, a near field communication (NFC) module, or other suitablecommunication module capable of communicating with another computingdevice. In various embodiments, the sensor devices, or a master sensordevice, may be configured to communicate the sensor data to a clientdevice asynchronously. The client device may include a hardwareprocessor, an operating system, and one or more applications(hereinafter client application or client applications) capable ofexecution on the client device by the hardware processor. In variousembodiments, the client application is configured to access the sensordata measured by the sensor devices. The sensor data may include theposition of the portion of the body during the movement performed.

In some embodiments, the client application is configured to generateand update a three-dimensional reconstruction of the movement performedfor viewing in a display, as will be described. Further, the clientapplication may be configured to determine whether the movement of thebody conforms to a predefined movement as well as identify a change inthe movement that, if performed, would conform to the predefinedmovement. In various embodiments, the client application may communicatethe sensor data to a remote computing device, such as a server, over anetwork for remote processing, as will be described.

As will be recognized herein, aspects of the embodiments havefar-reaching applications. For example, sensor devices may be used in aclinical setting to identify movements of an individual before and/orafter a medical procedure or diagnosis. The data may be used to evaluatethe mobility, stability, coordination of a patient, or other diagnosticneeds. Another example includes using the systems described herein intelemedicine applications, for example, where a medical professionalcreates predefined movements, analyzes performed movements, and performsdiagnoses and assignments remotely. Another possible use includes sportsand sports medicine, where an athlete is evaluated to improve techniqueor prevent injury. Further examples include employing the systemsdescribed herein to monitor movements of humanoid devices, animoiddevices, or other robots, for example, to improve gait technique duringrobotic development. This disclosure is not limited to the medical andsports fields or humans and may be extended to include analysis ofanimals or other animate systems.

In the following discussion, a general description of the system and itscomponents is provided, followed by a discussion of the operation of thesame.

With reference to FIG. 1, shown is a drawing of a body 100 of a personwith a plurality of sensor devices 103 a . . . 103 l (collectivelysensor devices 103) positioned thereon. In various embodiments, thesensor devices 103 may be secured to the body 100 using bands 106 a . .. 106 k (collectively bands 106), such as an elastic band that conformsto movement of the body 100 without detachment. Individual sensordevices 103 may be secured to the band 106, for example, by applying anadhesive to a sensor device 103, affixing the sensor device 103 to theband 106 using Velcro®, positioning the sensor device 103 in areceptacle of the band 106 (e.g., a pouch or a container), sewing thesensor device 103 to the band 106, or applying another suitable affixingtechnique. In some embodiments, the sensor device may be secured toclothing or adhered directly to the body.

For reference, the components of sensor device 103 a are shown inFIG. 1. As noted above, the sensor device 103 a may comprise circuitry,such as processing circuitry. In various embodiments, the sensor device103 a may comprise a microcontroller 109, an accelerometer 112, agyroscope 115, a magnetometer 117, a communication module 118, and/orother components. In further embodiments, the sensor device 103 a maycomprise other circuitry, such as an altimeter.

The microcontroller 109 may include circuitry, for example, on a singleintegrated circuit, such as a processor core, memory, and programmableinput and output peripherals. To this end, the microcontroller 109 maybe preconfigured to control the operations of the components of thesensor device 103 a, such as the accelerometer 112, the gyroscope 115,and/or the communication module 118. In some embodiments, themicrocontroller 109 performs control operations at the behest of aclient device 121 or a computing environment 124, as will be discussed.Further, the microcontroller 109 may be preconfigured to obtain andstore sensor data 127 a . . . 127 c (collectively sensor data 127) inlocal memory. The sensor data 127 a stored locally on the sensor device103 a may include, for example, position data collected from theaccelerometer 112, the gyroscope 115, as well as data received by thecommunication module 118.

The microcontroller 109 may be further configured to determine anorientation and position of the sensor device 103 a in athree-dimensional space, or raw sensor data 127 a may be communicated tothe client device 121 to determine the orientation or position. Thegyroscope 115 is configured to measure the rate of change of an angularposition of the sensor device 103 a over time, also referred to asangular velocity, in a unit of degrees per second. In other words, agyroscope 115 provides the derivative of the angular position over time:

$\begin{matrix}{\overset{.}{\theta} = \frac{d\theta}{dt}} & \left( {{eq}.\mspace{14mu} 1} \right)\end{matrix}$

where {dot over (θ)} is the metric obtained from the gyroscope 115,i.e., the angular position over time. As the metric obtained from thegyroscope 115 is the derivative of the angular position, the angularposition can be calculated by:

θ(t)=∫₀ ^(t){dot over (θ)}(t)dt≈Σ ₀ ^(T){dot over (θ)}(t)T _(s)  (eq. 2)

where θ(t) is the angular position at a given point in time, t, andT_(s) is a sampling frequency. In various embodiments, the samplingfrequency may be between 100 Hz and 200 Hz, although other samplingfrequencies may be employed. Sampling frequencies employed between 100Hz and 200 Hz may be ideal to eliminate errors caused by minor bumps andvibrations.

As the gyroscope 115 in some implementations may have difficultyreturning to zero when the sensor device 103 a returns to its originalposition, data from the accelerometer 112 may be utilized. In someembodiments, a filter may be applied to determine an angle for each axisthat may be used to ultimately calculate an orientation and position ofthe sensor device 103 a in a three-dimensional space. In someembodiments, a Kalman filter may be employed. In other embodiments, acomplimentary filter may be employed where an angle for a given axis maybe calculated via:

α=0.98*(α+Data_(Gyroscope) *dt)+0.02*(Data_(Accelerometer))  (eq. 3)

where α is the angle, 0.98 is the gravitational constant,Data_(Gyroscope) is data from a gyroscope 115, and Data_(Accelerometer)is data from an accelerometer 112.

In various embodiments, the components of the sensor device 103 a arecapable of measuring a movement having six-degrees-of-freedom (6DoF) ina three-dimensional space. In other words, each sensor device 103 maymonitor changes in its position, such as changes in left movement, rightmovement, upwards movements, downwards movement, forward movement,backwards movement, pitch, roll, and yaw. To this end, in someembodiments, the sensor device 103 a may comprise three accelerometers112 and three gyroscopes 115.

The communication module 118 includes a network interface componentcapable of sending and receiving data to other devices. To this end, thecommunication module 118 may include a Bluetooth® module, a Bluetooth®low-energy module, a ZigBee® module, a wireless fidelity (Wi-Fi) module,a near field communication (NFC) module, or other suitable module. Usingthe communication module 118, the microcontroller 109 may causetransmission of the sensor data 127 a from the sensor device 103 a to acomputing device, such as the client device 121 (e.g., as sensor data127 b).

A networked environment 130 is also described in accordance with variousembodiments. The networked environment 130 may be described as includingthe sensor devices 103, the computing environment 124, and the clientdevice 121. The computing environment 124 and the client device 121 arein data communication with each other via a network 133. The network 133includes, for example, the Internet, intranets, extranets, wide areanetworks (WANs), local area networks (LANs), wired networks, wirelessnetworks, or other suitable networks, etc., or any combination of two ormore such networks. For example, such networks may comprise satellitenetworks, cable networks, Ethernet networks, and other types ofnetworks.

The computing environment 124 may comprise, for example, a servercomputer or any other system providing computing capability.Alternatively, the computing environment 124 may employ a plurality ofcomputing devices that may be arranged, for example, in one or moreserver banks or computer banks or other arrangements. Such computingdevices may be located in a single installation or may be distributedamong many different geographical locations. For example, the computingenvironment 124 may include a plurality of computing devices thattogether may comprise a hosted computing resource, a grid computingresource and/or any other distributed computing arrangement. In somecases, the computing environment 124 may correspond to an elasticcomputing resource where the allotted capacity of processing, network,storage, or other computing-related resources may vary over time.

Various applications and/or other functionality may be executed in thecomputing environment 124 according to various embodiments. Also,various data is stored in a data store 136 that is accessible to thecomputing environment 124. The data store 136 may be representative of aplurality of data stores 136 as can be appreciated. The data stored inthe data store 136, for example, is associated with the operation of thevarious applications and/or functional entities described below.

The components executed on the computing environment 124, for example,include a data processing application 139, and other applications,services, processes, systems, engines, or functionality not discussed indetail herein. Generally, the data processing application 139 isexecuted to provide remote data services for sensor devices 103 andclient devices 121. To this end, in some embodiments, the dataprocessing application 139 is configured to receive and store sensordata 127 c from the sensor devices 103 via the client device 121 andperform remote processing on the sensor data 127. In some embodiments,the data processing application 139 may generate or updatethree-dimensional reconstructions of the body 100 for display in one ormore client devices 121. Further, the data processing application 139may be configured to permit a user to establish or update ideal movementdata 142 that may be used to determine, for example, whether a movementof the body 100 conforms to a predefined movement.

In one example, a medical professional may use her or her own batch ofsensor devices 103 to remotely define movements assigned to a patient.For example, the medical professional may attach sensor devices 103 tohis or her own body and perform a series of movements. The dataprocessing application 139 can monitor the movements performed by themedical professional and store the movements as ideal movement data 142.A patient may access the client application 158 to view the movementsprescribed to him or her by the medical professional, e.g., by viewing athree-dimensional reconstruction of the movement. The patient canattempt to perform the movement and the data processing application 139can determine whether the movement conforms to the ideal movement data142 created by the medical professional. Accordingly, the medicalprofessional can determine (a) whether the patient has performed his orher assigned movements; (b) whether the movements performed by the userconform to the movement assigned by the medical professional; and (c) ifthe movement performed by the patient does not conform to the movementassigned by the medical professional, determine why the movementperformed by the patient does not conform to the movement assigned bythe medical professional.

In addition, the data processing application 139 can identify whymovements performed by a user do not conform to ideal movement data 142and generate suggested changes 145 to the movements that, if performed,might cause the movements to conform to ideal movement data 142.

The data stored in the data store 136 includes, for example, idealmovement data 142, suggested changes 145, sensor arrangements 148,three-dimensional reconstructions 150 a . . . 150 b (collectivelythree-dimensional reconstructions 150), user data 152, and potentiallyother data. The ideal movement data 142 may include sensor datacollected during an ideal movement performed by a body 100. For example,a professional basketball player may wear sensor devices 103 whileperforming a jump shot, a professional baseball player may wear sensordevices 103 while pitching, or a professional pole vaulter may wearsensor devices 103 while performing an ideal jump. Sensor data 127 maybe collected during these ideal movements and stored in the data store136 as ideal movement data 142.

In other embodiments, a user may define ideal movement data 142 to beperformed by another user. For example, a medical professional maydefine ideal movement data 142 for a patient recently having a ulnarcollateral ligament reconstruction (UCL) treatment. The ideal movementdata 142 may include data collected from sensor devices 103 when themedical professional, or other body 100, performs movements, such asrotating the shoulder up, down, left, and right, simulating a pitchingmotion, etc. The patient may be assigned the movements to be performedand the actual movements performed by the patient may be compared to theideal movement data 142.

The suggested changes 145 includes data indicative of why movementsperformed by a user do not conform to ideal movement data 142 as well assuggested movements that, if performed, might cause the movements toconform to ideal movement data 142. Moving along, sensor arrangements148 may include one or more predefined arrangements of sensor devices103 capable of correlating a particular sensor to a particular positionon a body 100. An example database table of a predefined arrangement ofsensor devices 103 that may be stored in the data store 136 is shownbelow in Table 1:

TABLE 1 Example Database Table of a Predefined Arrangement of SensorDevices 103 int Sensor_Identifier var Position int User_ID 012333457Left Wrist 042857 012333458 Right Wrist 042857 012333459 Left Elbow042857 . . . . . . . . .

As shown above in Table 1, a sensor device 103 may be associated with asensor identifier, which may include a unique identifier. The databasetable may be queried using the sensor identifier to identify theposition of the corresponding sensor device 103 on the body 100.Further, the sensor identifier may be stored in association with a useridentifier, which may include an identifier that uniquely identifies auser of the client device 121, e.g., the wearer of the sensor devices103. Accordingly, different arrangements of sensor devices 103 may beassigned to different bodies 100, whether persons, animals, humanoids,animoids, etc. In various embodiments, the sensor arrangements 148 maybe predefined by a user of the client device 121 (e.g., a wearer of thesensor devices 103) or by another user (e.g., a medical professional, acoach, or other person).

The three-dimensional reconstructions 150 include graphicalrepresentations of an electronically generated body performing amovement that may be generated by the client device 121 or the computingenvironment 124. In various embodiments, the three-dimensionalreconstructions 150 may be generated by a three-dimensional model andfigure generator in the form of an application, service, or library. Inembodiments where an application generates the three-dimensionalreconstruction 150 comprises Poser®, MakeHuman™, DAZ 3D, Misfit Model3D, Mixamo® Fuse, XnaLara Realtime Posing Program, GLLara, or Vizago.

User data 152 may include data corresponding to the wearer of the sensordevices 103 which may be the user of the client device 121. To this end,the sensor data 127 c collected during a movement performed by a usermay be stored in association with the user as well as a time themovement occurred.

The client device 121 is representative of a plurality of client devices121 that may be coupled to the network 133 The client device 121 maycomprise, for example, a processor-based system such as a computersystem. Such a computer system may be embodied in the form of a desktopcomputer, a laptop computer, personal digital assistants, cellulartelephones, smartphones, set-top boxes, music players, web pads, tabletcomputer systems, game consoles, electronic book readers, or otherdevices with like capability. The client device 121 may include adisplay 155. The display 155 may comprise, for example, one or moredevices such as liquid crystal display (LCD) displays, gas plasma-basedflat panel displays, organic light emitting diode (OLED) displays,electrophoretic ink (E ink) displays, LCD projectors, or other types ofdisplay devices, etc.

The client device 121 may be configured to execute various applicationssuch as a client application 158 and/or other applications. The clientapplication 158 may be executed in a client device 121, for example, toaccess content served up by the computing environment 124, otherservers, and/or the sensor devices 103 to render a user interface 169 onthe display 155. To this end, the client application 158 may comprise,for example, a dedicated application, such as a mobile or clientapplication, a web browser, etc., and the user interface 169 maycomprise an application screen, a network page, etc. The client device121 may be configured to execute applications beyond the clientapplication 158 such as, for example, email applications, socialnetworking applications, word processors, spreadsheets, and/or otherapplications.

Referring next to FIG. 2, shown is a drawing of a body 100 performing amovement where sensor data 127 is collected during the movement andcommunicated to a client device 121. The client application 158 may beexecuted in the client device 121 to access and store the sensor data127. In various embodiments, the client application 158 may beconfigured to generate a three-dimensional reconstruction 150 a of themovement in near-real-time as the movement is performed. To this end, athree-dimensional reconstruction 150 a may be rendered in the display155 as the movement is performed, or thereafter.

As may be appreciated, the three-dimensional reconstruction 150 a maynot include a video feed of the body 100 performing the movement.Instead, the three-dimensional reconstruction 150 a includes graphicalrepresentations of an electronically generated body performing amovement. In one embodiment, a single three-dimensional reconstruction150 a may be shown in the display 155 that mirrors the movementsperformed by the body 100. In another embodiment, a firstthree-dimensional reconstruction 150 a is shown along a secondthree-dimensional reconstruction 150 b. For example, the firstthree-dimensional reconstruction 150 a may reflect a movement performedby the body 100 while the second three-dimensional reconstruction 150 breflects a reconstruction of ideal movement data 142.

To this end, the movement performed by the user may be compared to amovement performed by himself or herself in the past or may be comparedto a movement performed by another person, such as a movement assignedperformed by a medical professional when assigning a movement or amovement performed by an athlete. The first three-dimensionalreconstruction 150 a may be overlaid on the second three-dimensionalreconstruction 150 b, or vice versa. In other embodiments, the firstthree-dimensional reconstruction 150 a and the second three-dimensionalreconstruction 150 b may be shown side by side or one or top of theother in the display 155.

Moving on to FIG. 3, shown is another example of a client device 121that may be used to display a three-dimensional reconstruction while amovement is performed or thereafter. In one embodiment, the clientdevice 121 may comprise a smartwatch that may show one or morethree-dimensional reconstructions 150 in the display 155 that mirrorsthe movements performed by the body 100.

Turning now to FIG. 4, shown is another drawing of a body 100 performinga movement where sensor data 127 is collected during the movement andcommunicated to a client device 121. In the non-limiting example of FIG.4, a golf swing is shown as a movement that may be performed to collectsensor data 127 during the movement. The sampling period to collectsensor data 127 may include a start of the golf swing and an end of thegolf swing, as may be appreciated. The sampling rate may include apredefined time interval at which a position and/or orientation isdetermined and/or obtained from each of the sensor devices 103. Forexample, a position and/or orientation of the sensor devices 103 may bedetermined and/or obtained every 0.5 ms, or other suitable timeinterval.

As shown in FIG. 5, the sensor data 127 collected during the golf swingmovement may be used to render a three-dimensional reconstruction 150 ain near-real-time for encoding in a user interface 169 rendered on adisplay 155, either as the movement is performed or thereafter. To thisend, an athlete is able to view his or her movement to improvetechnique. In the non-limiting example of FIG. 5, the client application158 causes a rendering of a first three-dimensional reconstruction 150 aside-by-side with a second three-dimensional reconstruction 150 b. Thefirst three-dimensional reconstruction 150 a may be generated using thesensor data 127 collected during the movement while the secondthree-dimensional reconstruction 150 b may be generated using idealmovement data 142.

The client application 158 may be configured to analyze differencesbetween the sensor data 127 collected during the movement and the idealmovement data 142. As noted above, the ideal movement data 142 mayreflect sensor data 127 collected by a person who performed an idealmovement. Accordingly, a comparison of sensor data 127 to sensor data127 is performed. In other words, sensor data 127 collected when a firstperson performed a movement is compared to sensor data 127 collectedwhen a second person performed the same movement. A position of a sensordevice 103 on the first person may be compared to a position of a sensordevice 103 on a second person during at a same point in time during themovement. For example, given the time t, a position of sensor device 103for a first person may be compared to a position of a sensor device 103for a second person and, if the differences in position exceed apredefined threshold, a user of the client device 121 may be notified ofthe difference.

As a non-limiting example, a wrist position of a professional golfer maybe monitored during a golf swing by placing a sensor device 103 at thewrist of the professional golfer. Sensor data 127 is collected from thesensor device 103 during the golf swing to measure a position andorientation of the sensor device 103 (i.e., the wrist) at a predefinedsampling rate. This data may be stored as ideal movement data 142.Similarly, a wrist position of an amateur golfer may be monitored duringa golf swing by placing a sensor device 103 at the wrist of the amateurgolfer. Sensor data 127 is also collected from the sensor device 103during the golf swing to measure a position and orientation of thesensor device 103 at the same predefined sampling rate. The sensor data127 collected from the amateur golfer may be compared to that of theprofessional golfer to identify differences between the positions of thewrist during the golf swing. If a difference in the position exceeds apredefined threshold (e.g., that allows for some degree of error), theuser of the client device 121 may be notified. Further, a suggestedchange 145 may be shown to the user of the client device 121.

A function may be used to obtain a position (and/or orientation) of asensor device 103 at a given time:

float sensor_position(int time, int device_id) { // Establish Connectionwith Database $conn = new mysqli([AUTHENTICATION_INFO]); // QueryDatabase to Find Position $query_string = “SELECT * FROM table WHEREdevice_id=’”+$device_id+”’ AND time=’”+$time+”’”; $position =$conn−>query($query_string); // Return Position return(position); }

To this end, a first position of a first sensor device 103 (e.g., forthe amateur golfer) may be obtained using the function and a secondposition of a second sensor device 103 (e.g., for the professionalgolfer). The positions may then be compared. If the first sensor device103 is within a predefined threshold, the position of the amateur golferis nearly identical to that of the professional golfer. However, if thefirst sensor device 103 is not within a predefined threshold, theposition of the amateur golfer is not identical to that of theprofessional golfer. Accordingly, the user of the client device 121 maybe notified to adjust his or her wrist position at a given point in timeduring the swing using a suggested change 145.

In various embodiments, a time at which a position of a first group ofsensor devices 103 (e.g., for a first person) is compared to a secondgroup of sensor device 103 (e.g., for a second person) may be specifiedby a slider bar 503. As a result, a user of the client device 121 mayaccess and view suggested changes 145 during a frame-by-frame analysisof the movement. Alternatively, the user may view the movement in itsentirety at a real-time play speed to see suggested changes 145 atdifferent times during playback of the movement. In some embodiments,the playback of the movement may be sped up or slowed down.

Further, the client application 158 may be executed in the client device121 and may store the sensor data 127 for later access and/orcommunicate the sensor data 127 over the network 133 to be storedremotely by the computing environment 124. The client application 158may be configured to generate the three-dimensional reconstructions 150or the computing environment 124 may remotely generate thethree-dimensional reconstruction 150 and provide them to the clientdevice 121 for rendering in the display 155.

As may be appreciated, a three-dimensional reconstruction 150 may notinclude a video feed of the body 100 performing the movement. Instead,the three-dimensional reconstructions 150 include graphicalrepresentations of an electronically generated body performing amovement. In one embodiment, a single three-dimensional reconstruction150 may be shown in the display 155 that mirrors the movements performedby the body 100. In another embodiment, a first three-dimensionalreconstruction 150 a is shown along a second three-dimensionalreconstruction 150 b. For example, the first three-dimensionalreconstruction 150 a may reflect a movement performed by the body 100while the second three-dimensional reconstruction 150 b reflects areconstruction of ideal movement data 142. To this end, the movementperformed by the user may be compared to a movement performed by himselfor herself in the past or may be compared to a movement performed byanother person, such as a movement assigned performed by a medicalprofessional when assigning a movement or a movement performed by anathlete.

Moving on to FIG. 6, shown is a drawing of a client device 121 used torender three-dimensional reconstructions 150 a . . . 150 b. In variouscircumstances, a person may have a surgery and, as a result of thesurgery, a movement performed by the person is changed. For example, aperson might walk, run, or perform other exercises differently after aknee surgery.

Accordingly, in various embodiments, the client application 158 may beexecuted to compare sensor data 127 collected for a patient during amovement (e.g., gait, running, or other movement) to sensor data 127collected by a representative patient to show expected differencesbetween the movements. For example, a notification may be presented inthe client application 158 that indicates that knee flexibility will beimproved.

Similarly, in FIG. 7, the client application 158 may be executed tocompare sensor data 127 collected by a medical professional to remotelyassign a movement to a patient. The sensor data 127 may be collected bythe medical professional wearing sensor devices 103 or the sensor data127 may be accessed from a database of previously analyzed movements.For example, a medical professional may use sensor data 127 collectedfrom another patient to assign a movement to a patient.

Either the patient and/or the medical professional may use a clientdevice 121 to access three-dimensional reconstructions 150 generatedusing the sensor data 127. As a result, the exercise performed by apatient may be compared to the assigned exercise to see if the patientcan conform his or her movement to that of the assigned exercise. Theclient application 158 may detect differences between the two sets ofsensor data 127 and communicate the differences to the patient and/orthe medical profession, for example, by rendering a suitablenotification in the user interface 169. Additionally, notifications fromthe doctor to patient, or vice versa, may be rendered in the display155, for example, to direct the patient how to improve or to notify thepatient that he or she is performing well.

Referring next to FIG. 8, shown is a flowchart that provides one exampleof the operation of a portion of the microcontroller 109 of the sensordevice 103 according to various embodiments. It is understood that theflowchart of FIG. 8 provides merely an example of the many differenttypes of functional arrangements that may be employed to implement theoperation of the portion of the microcontroller 109 as described herein.As an alternative, the flowchart of FIG. 8 may be viewed as depicting anexample of elements of a method implemented by a microprocessorapplication executable by a hardware processor according to one or moreembodiments.

Beginning with 803, the microcontroller 109 initializes its serial portsor other communication interface, if necessary. For example, somemicrocontrollers 109 communicate with components of the microcontroller109, or other circuitry, using serial port processing. In oneembodiment, the microcontroller 109 communicates with the one or moreaccelerometers 112, gyroscopes 115, magnetometers 117, etc., usingserial ports. To this end, in some embodiments, a microprocessorapplication may perform a function call to a library to initialize theserial ports for communication.

Next, in 806, sensor data 127 is collected from the one or moregyroscopes 115, accelerometers 112, magnetometers 117, etc. Inembodiments where the sensor device 103 is capable of 6DoF, sensor data127 is collected from three gyroscopes 115 and three accelerometers 112.In other embodiments, the sensor device may be capable of 9DoF or more.

Next, in 809, the microcontroller 109 may process the sensor data 127,for example, to determine an orientation and position of the sensordevice 103 in a three-dimensional space. In alternative embodiments, rawsensor data 127 a may be communicated to the client device 121 todetermine the orientation or position remotely. If the microcontroller109 locally processes the sensor data, in some embodiments, themicrocontroller 109 ultimately may determine four quaternions fromsensor data 127 collected from an inertial measurement unit (IMU) thatmay comprise the gyroscopes 115, the accelerometers 112, and/or otherdevices. The output data from the sensors could be quaternion vectors ora three-dimensional position (x, y, z) with an angle of rotation toestablish a rotational matrix. Each quaternion may include afour-element vector that can be used to encode a rotation in athree-dimensional coordinate system. As may be appreciated, thequaternions may be used to calculate orientation and position (orlocation) in a three-dimensional coordinate system and utilized assuitable input to graphics code for three-dimensional reconstruction

In 812, a determination may be made whether the sensor device 103 onwhich the microcontroller 109 is embedded is a master sensor device. Asnoted above, in some embodiments, at least one of the sensor devices 103in a group of sensor devices 103 may be dedicated as a “master” sensordevice 103, where other ones of the sensor devices 103 are “slave”sensor devices 103. Accordingly, if the sensor device is not designatedas a master sensor device 103, the process proceeds to 815 where thesensor data 127 (as processed) is communicated to a master sensor device103. Thereafter, the process proceeds to completion.

Referring back to 812, if the sensor device 103 on which themicrocontroller 109 is embedded is designated as the master sensordevice the process proceeds to 818 where sensor data 127 is collectedfrom the slave sensor devices 103. Next, in 821, the sensor data 127collected by the components of the master and slave sensor devices 103is communicated to a client device 121, such as a smartphone, a tabletcomputing device, a laptop computing device, a smartwatch, or othercomputing device.

As noted above, the sensor devices 103 include a communication module118, such as a network interface component capable of sending andreceiving data to other devices. To this end, the communication module118 may include a Bluetooth® module, a Bluetooth® low-energy module, aZigBee® module, a Wi-Fi module, a near field communication (NFC) module,or other suitable module. Using the communication module 118, themicrocontroller 109 may cause transmission of the sensor data 127 fromthe sensor device 103 to a computing device, such as the client device121. In embodiments where the communication module 118 includes a Wi-Fimodule, the sensor data 127 may be communicated directly to thecomputing environment 124 using a wireless network.

Turning now to FIG. 9, shown is an example of pseudocode that may beused to configure a microprocessor application executable by a hardwareprocessor of a microcontroller 109 onboard a sensor device 103. Thepseudocode reflects an example of an algorithm that causes performanceof some functions of the sensor device 103 (or the microcontroller 109of the sensor device 103) as described throughout the presentdisclosure. The pseudocode may not reflect a particular softwarelanguage, but may describe functions, function calls, variabledefinitions, or other actions that, if implemented in one or moresoftware languages, configure the microcontroller 109 to perform thefunctions described herein.

Notably, the pseudocode describes including applicable libraries andcreating a variable for device identifier data and quaternion data,which may be embodied as an array of floats. A library included in theapplication may comprise classes, functions, or other calls that permithardware control. For example, an object for “SensorDevice” may becreated where .Initialize( ) is a function that initializes the sensordevice.

In a setup routine, serial ports or other communication interfaces maybe initialized, if necessary. For example, a microcontroller 109 may bemanufactured to communicate with other devices or circuitry using serialport processing. The microprocessor application may perform a functioncall to a suitable library to initialize the serial ports forcommunication. A wait is performed to allow the serial ports toinitialize.

Thereafter, a loop routine is performed to collect data from componentsof the sensor device 103, such as the IMU. For example, a sensoridentifier may be accessed from the IMU and stored at location [0] inarray q, a first quaternion may be accessed from the IMU and stored atlocation [1] in array q, and so forth. The array q is then communicatedover the serial port. In one example, the serial port allows the data tobe communicated to the communication module 118, which then communicatesthe data to the client device 121. The loop continues to repeat untilthe application is exited or terminated based on a predefined condition(e.g., powering off the device, disabling data collection, or othercondition).

Moving on to FIG. 10, shown is a flowchart that provides one example ofthe operation of a portion of the client device 121 according to variousembodiments. It is understood that the flowchart of FIG. 10 providesmerely an example of the many different types of functional arrangementsthat may be employed to implement the operation of the portion of theclient device 121 as described herein. As an alternative, the flowchartof FIG. 10 may be viewed as depicting an example of elements of a methodimplemented by the client application 158 executable by a hardwareprocessor according to one or more embodiments.

As noted above, one or more sensor devices 103 can be positioned on abody 100 of a person, animal, humanoid, animoid, or other moveable bodyto accurately track movement where more sensor devices 103 can producemore granularities in movement detection. Each sensor device 103 may becoupled to a band 106 or other attachment that secures the sensor device103 to a position on the body 100. Additionally, each sensor device 103may include circuitry that measures a position of the portion of thebody 100 to which the band is secured during a movement performed by theportion of the body 100. As shown in FIG. 1, each sensor device 103 maycomprise a microcontroller 109, an accelerometer 112, a gyroscope 115, amagnetometer 117, a communication module 118, and/or other components asmay be appreciated. As at least one of the sensor devices 103 isconfigured to communicate with a client device 121, the clientapplication 158 can process sensor data 127 and generate user interfaces169 that include three-dimensional reconstructions 150 of performedmovements.

The sensor devices 103 may be placed in accordance with a predefinedarrangement stored in memory (e.g., the data store 136), such that acomputing device can correlate each of the sensor devices 103 to arespective position of a body 100. In one embodiment, assuming the body100 is that of a person or a humanoid, a first sensor device 103 a maybe positioned at a first shoulder of the body 100; a second sensordevice 103 b may be positioned at a second shoulder of the body 100; athird sensor device 103 c may be positioned at a first elbow of the body100; a fourth sensor device 103 d may be positioned at a second elbow ofthe body 100; a fifth sensor device 103 e may be positioned at a firstwrist of the body 100; a sixth sensor device 103 f may be positioned ata second wrist of the body 100; a seventh sensor device 103 g may bepositioned at a first knee of the body 100; an eighth sensor device 103h may be positioned at a second knee of the body 100; a ninth sensordevice 103 i may be positioned at a first ankle of the body 100; a tenthsensor device 103 j may be positioned at a second ankle of the body 100;an eleventh sensor device 103 k may be positioned at a first side ofhips of the body 100; and a twelfth sensor device 103 l may bepositioned at a second side of hips of the body 100.

Beginning with 1003, the client device 121 enables the sensor devices103 to collect sensor data 127. In one embodiment, the client device 121sends a suitable control signal to each sensor device 103 that causesthe sensor device 103 to start collecting sensor data 127. In anotherembodiment, the client device 121 sends a suitable control signal to amaster sensor device 103, which then sends suitable control signals toslave sensor devices 103 that causes the sensor devices 103 to startcollecting sensor data 127.

Next, in 1006, the client device 121 initializes the display of one ormore three-dimensional reconstructions 150. To this end, a clientapplication 158 or service may be executed that utilizes graphiclibraries to eventually render a three-dimensional reconstruction 150 ofa body 100, where the three-dimensional reconstruction 150 is updated innear-real-time to mirror movements performed by the body 100 wearing thesensor devices 103 using the three-dimensional reconstruction 150.

In 1009, sensor data 127 is received from the sensor devices 103, forexample, using Bluetooth®, ZigBee®, Wi-Fi, or other suitablecommunication medium. As noted above, the sensor devices 103 may includea communication module 118, such as a Bluetooth® module, a Bluetooth®low-energy module, a ZigBee® module, an NFC module, a Wi-Fi module, orother suitable communication module 118 capable of communicating withanother computing device. In various embodiments, the sensor devices103, or a master sensor device 103, may be configured to communicate thesensor data 127 to a client device 121 asynchronously. The client device121 may include a hardware processor, an operating system, and one ormore applications (hereinafter client application or clientapplications) capable of execution on the client device by the hardwareprocessor. In various embodiments, the client application 158 isconfigured to access the sensor data 127 measured by the sensor devices.The sensor data 127 may include the position of the portion of the body100 during the movement performed, or other data collection during themovement.

In 1012, the sensor data 127 collected from the sensor devices 103 maybe stored in a buffer or in local memory. To this end, in someembodiments, the sensor data 127 may be stored in a database (e.g.,structured query language (SQL) database or SQL-Lite database) of thedata store 136 in association with a time the movement was performed.

Next, in 1015, the sensor data 127 may be processed to determine aposition and/or orientation of each of the sensors devices 103 based onthe sensor data 127 received from each of the sensor devices 103. Forexample, sensor data 127 received from a sensor device 103 at a shouldermay be processed to identify a position or orientation of the sensordevice 103 at the shoulder, and so forth. In some embodiments, thesensor data 127 is processed to monitor changes in a position of acorresponding sensor device 103, such as changes in left movement, rightmovement, upwards movements, downwards movement, forward movement,backwards movement, pitch, roll, and yaw.

Using the processed sensor data 127, in 1018, the three-dimensionalreconstruction 150 is generated, updated, and/or rendered in a display155 of the client device 121. In one embodiment, to update thethree-dimensional reconstruction 150, a programmatic function call orapplication programming interface (API) call is performed to provide agraphics library with an updated position and/or orientation of thecorresponding portion of the body 100.

Further, in some embodiments, the client device 121 (e.g., via theclient application 158) may be configured to determine whether amovement performed by a body 100, having the sensor devices 103 coupledthereon, conforms to an ideal movement, such as a movement assigned by amedical professional, a movement performed by a coach or athlete, orother predefined movement. Accordingly, in 1021, the movement performedby the body 100 having the sensor devices 103 coupled thereon iscompared to ideal movement data 142 representative of an ideal movement.

The ideal movement data 142 may reflect sensor data 127 collected by aperson who performed an ideal movement. As a result, a comparison ofsensor data 127 to sensor data 127 is performed. In other words, sensordata 127 collected when a first person performed a movement is comparedto sensor data 127 collected when a second person performed the samemovement. A position of a sensor device 103 on the first person may becompared to a position of a sensor device 103 on a second person duringat a same point in time during the movement. For example, given the timet, a position of sensor device 103 for a first person may be compared toa position of a sensor device 103 for a second person and, if thedifferences in position exceed a predefined threshold, a user of theclient device 121 may be notified of the difference. As a non-limitingexample, given t, a query is performed to obtain data from the idealmovement data 142.

-   -   SELECT * FROM ideal_movement_data_table WHERE        sensor_id=‘[SENSORID]’ AND time=‘[TIME t]”

Similarly, given the same time t, a query is performed to obtain datafrom the sensor devices 103 performed by the body 100.

-   -   SELECT * FROM sensor_data_table WHERE sensor_id=‘[SENSORID]’ AND        time=‘[TIME t]”

As a non-limiting example, Table 2 describes the data from the query tothe “ideal_movement_data_table” and the query to the“sensor_data_table:”

TABLE 2 Sample Results from Two Queries to Compare Sensor PositionsSensor_ID Sensor_Pos User_ID Time Q1 . . . Q4 012333457 Left Wrist042857 t [Q_VALUES] 489611342 Left Wrist 054621 t [Q_VALUES]The row for “sensor_id”=01233457 contains information associated with anideal movement data 142 and the row for “sensor_id”=489611342 containsinformation associated with sensor data 127 obtained from a movementperformed, for example, by a current wearer of the sensor device 103.The values of the two rows stored as columns under “pos” may be comparedto determine whether a first movement conforms to a second movement, orvice versa. The comparison may include accounting for error using apredefined threshold, as may be appreciated.

In 1024, a determination is made whether a first movement (e.g., amovement performed by a wearer of the sensor devices 103) conforms to asecond movement (e.g., ideal movement data 142). If a first movement iswithin a predefined threshold of a second movement, no action may betaken and the process may proceed to 1027. However, in some embodiments,a notification may be generated for rendering on a display 155 thatindicates that the movement performed by a user conforms with idealmovement data 142 (e.g., “Good job, your movement is correct!”).

Referring back to 1024, if the first movement does not conform to thesecond movement (e.g., the first position is not within a predefinedthreshold of the second movement), the process proceeds to 1030 where asuggested change 145 in the movement may be generated for rendering inthe display 155. The suggested change 145 may include, for example, achange in the position of the portion of the body 100 where a particularsensor device 103 is attached that, if performed as suggested, wouldconform to the ideal movement data 142. As may be appreciated, the userof the client device 121 may then attempt to perform a subsequentmovement based on the suggested change 145 to conform to an idealmovement.

The process then proceeds to 1027 where a determination is made whethera terminate condition has been identified. In some embodiments, aterminate condition may occur after a predefined amount of time, after acomplete movement has been performed (e.g., a complete pitch motion, acomplete jump shot, or a complete golf swing), when a user closes theclient application 158, or when a user pauses the collection of sensordata 127 in the client application 158. If a terminate condition is notidentified, the process reverts back to 1009 to continue accessingcollected sensor data 127, for example, to continue updating thethree-dimensional reconstruction 150 and to continue generatingsuggested changes 145 in movement. If a terminate condition isidentified, however, the process may proceed to completion.

Referring next to FIG. 11, shown is an example of pseudocode that may beused to configure the client application 158 ultimately executable by ahardware processor of the client device 121. The pseudocode reflects anexample of an algorithm that causes performance of some functions of theclient device 121 as described throughout the present disclosure. Thepseudocode may not reflect a particular software language, but maydescribe functions, function calls, variable definitions, or otheractions that, if implemented in one or more software languages,configure the client device 121 to perform the functions describedherein.

Notably, the pseudocode describes including applicable libraries toallow access to particular libraries that may be needed to interfacewith particular peripherals of the client device 121, such as thedisplay 155, the serial ports or other communication modules, memory,etc. Various data objects (e.g., JavaScript Object Notation (JSON) dataobjects), screen dimensions, serial ports, and variables may beinitialized. Further, an object model may be initialized to create athree-dimensional reconstruction 150 using a suitable graphics library.In some embodiments, node.js library can be used to generate or modify athree-dimensional reconstruction.

A setup routine may be performed that establishes a size of a graphicaluser interface (e.g., user interface 169), loads images to be used inobject models (e.g., three-dimensional reconstructions), scales theimages to be used in the object models, defines a user interface area(e.g., a bounding box), instantiates serial ports, and stores dataobtained over the serial ports (e.g., sensor data 127 obtained from oneor more of the sensor devices 103) in a buffer.

A serial event routine may be performed that identifies which one of thesensor devices 103 transmitted sensor data 127 received by the clientdevice 121 (e.g., using a unique sensor identifier). The sensor data 127may be stored in association with the sensor device 103 thatcommunicated the sensor data 127 as well as a time the sensor data 127was obtained. A call to a quaternion processing routine may be called toprocess quaternions received from a particular sensor device 103.

The quaternion processing routine may be called to create a string ofcharacters of the sensor data 127 received from the particular sensordevice 103. As the sensor data 127 is received from the sensor device103 as a stream of characters, the characters may be stored in a stringdata object and parsed using a “split( )” function. The quaternionsobtained from the particular sensor device 103 may be casted into floatsto allow the client application 158 to perform subsequent processingwhich might not be available were the quaternions stored as strings ofcharacters. An axis is identified from the quaternions as well as anangle of rotation. Using the axis and the angle of rotation, a functioncall may be performed to a “Draw( ) routine” that generates and/orupdates a three-dimensional reconstruction 150.

The “Draw( ) routine” may include a loop to constantly update thethree-dimensional reconstruction 150 until a terminate condition isidentifier. The routine may push orientation matrices that updateapplicable portions of the body 100 as well as rotate the orientationmatrices by the determined angle and axis of rotation.

Moving on to FIG. 12, shown is a flowchart that provides one example ofthe operation of a portion of the computing environment 124 according tovarious embodiments. It is understood that the flowchart of FIG. 12provides merely an example of the many different types of functionalarrangements that may be employed to implement the operation of theportion of the client device 121 as described herein. As an alternative,the flowchart of FIG. 12 may be viewed as depicting an example ofelements of a method implemented by the data processing application 139executable by a hardware processor of the computing environment 124according to one or more embodiments.

In various embodiments, the functions performed by the clientapplication 158 can be offloaded for processing by the computingenvironment 124. Beginning with 1203, the computing environment 124accesses a request received from the client device 121 for athree-dimensional reconstruction 150. In some embodiments, the requestmay include sensor data 127 received by the client device 121 from thesensor devices 103. Accordingly, in 1206, the computing environment 124accesses sensor data 127 received by the computing environment 124 from,for example, the client device 121 and the sensor devices 103.

As noted above, the sensor devices 103 may include a communicationmodule 118, such as a Bluetooth® module, a Bluetooth® low-energy module,a ZigBee® module, an NFC module, a Wi-Fi module, or other suitablecommunication module 118 capable of communicating with another computingdevice. In various embodiments, the sensor devices 103, or a mastersensor device 103, may be configured to communicate the sensor data 127to the computing environment 124 asynchronously. The data processingapplication 139 may be configured to access the sensor data 127 measuredby the sensor devices 103. The sensor data 127 may include the positionof the portion of the body 100 during the movement performed, or otherdata collected during the movement.

In 1209, the sensor data 127 collected from the client device 121 or thesensor devices 103 may be stored in a buffer or in the data store 136.To this end, in some embodiments, the sensor data 127 may be stored in adatabase (e.g., SQL database or SQL-Lite database) of the data store 136in association with a time the movement was performed.

Next, in 1212, the sensor data 127 may be processed to determine aposition and/or orientation of each of the sensors devices 103 based onthe sensor data 127 received from each of the sensor devices 103. Forexample, sensor data 127 received from a sensor device 103 at a shouldermay be processed to identify a position or orientation of the sensordevice 103 at the shoulder, and so forth. In some embodiments, thesensor data 127 is processed to monitor changes in a position of acorresponding sensor device 103, such as changes in left movement, rightmovement, upwards movements, downwards movement, forward movement,backwards movement, pitch, roll, and yaw.

Using the processed sensor data 127, in 1218, the three-dimensionalreconstruction 150 is generated or updated and is transmitted to theclient device 121 over the network 133 for rendering in the display 155.In one embodiment, to update the three-dimensional reconstruction 150, aprogrammatic function call or API call is performed to provide agraphics library with an updated position and/or orientation of thecorresponding portion of the body 100.

Further, in some embodiments, the computing environment 124 (e.g., viathe data processing application 139) may be configured to determinewhether a movement performed by a body 100, having the sensor devices103 coupled thereon, conforms to an ideal movement, such as a movementassigned by a medical professional, a movement performed by a coach orathlete, or other predefined movement. Accordingly, in 1221, themovement performed by the body 100 having the sensor devices 103 coupledthereon is compared to ideal movement data 142 representative of anideal movement.

The ideal movement data 142 may reflect sensor data 127 collected by aperson who performed an ideal movement. As a result, a comparison ofsensor data 127 to sensor data 127 is performed. In other words, sensordata 127 collected when a first person performed a movement is comparedto sensor data 127 collected when a second person performed the samemovement. A position of a sensor device 103 on the first person may becompared to a position of a sensor device 103 on a second person duringat a same point in time during the movement.

In 1224, a determination is made whether a first movement (e.g., amovement performed by a wearer of the sensor devices 103) conforms to asecond movement (e.g., ideal movement data 142). If a first movement iswithin a predefined threshold of a second movement, no action may betaken and the process may proceed to 1227. However, in some embodiments,a notification may be generated for rendering on a display 155 thatindicates that the movement performed by a user conforms with idealmovement data 142 (e.g., “Good job, your movement is correct!”).

Referring back to 1224, if the first movement does not conform to thesecond movement (e.g., the first position is not within a predefinedthreshold of the second movement), the process proceeds to 1230 where asuggested change 145 in the movement may be generated for rendering inthe display 155. The suggested change 145 may include, for example, achange in the position of the portion of the body 100 where a particularsensor device 103 is attached that, if performed as suggested, wouldconform to the ideal movement data 142. As may be appreciated, the userof the client device 121 may then attempt to perform a subsequentmovement based on the suggested change 145 to conform to an idealmovement.

The process then proceeds to 1227 where a determination is made whethera terminate condition has been identified. In some embodiments, aterminate condition may occur after a predefined amount of time, after acomplete movement has been performed (e.g., a complete pitch motion, acomplete jump shot, or a complete golf swing), when a user closes theclient application 158, or when a user pauses the collection of sensordata 127 in the client application 158. If a terminate condition is notidentified, the process reverts back to 1009 to continue accessingcollected sensor data 127, for example, to continue updating thethree-dimensional reconstruction 150 and to continue generatingsuggested changes 145 in movement. If a terminate condition isidentified, however, the process may proceed to completion.

With reference to FIG. 13, shown is a schematic block diagram of asensor device 103 according to an embodiment of the present disclosure.The computing environment 124 includes one or more microcontroller 109.Each microcontroller 109 includes at least one processor circuit, forexample, having a processor 1303 and a memory 1306, both of which arecoupled to a local interface 1309. The local interface 1309 maycomprise, for example, a data bus with an accompanying address/controlbus or other bus structure as can be appreciated. The sensor device 103may further comprise one or more accelerometers 112, gyroscopes 115,magnetometers 117, and/or other components.

Stored in the memory 1306 are both data and several components that areexecutable by the processor 1303. In particular, stored in the memory1306 and executable by the processor 1303 is a microcontrollerapplication 1312, and potentially other applications. Also stored in thememory 1306 may be a sensor data store 1315 and other data. In addition,an operating system may be stored in the memory 1306 and executable bythe processor 1303.

Turning now to FIG. 14, shown is a schematic block diagram of the clientdevice 121 according to an embodiment of the present disclosure. Theclient device 121 includes one or more computing devices. Each computingdevice includes at least one processor circuit, for example, having aprocessor 1403 and a memory 1406, both of which are coupled to a localinterface 1409. To this end, each computing device may comprise, forexample, a mobile computing device, such as a smartphone, a tablet, asmartwatch, or other similar device. The local interface 1409 maycomprise, for example, a data bus with an accompanying address/controlbus or other bus structure as can be appreciated.

Stored in the memory 1406 are both data and several components that areexecutable by the processor 1403. In particular, stored in the memory1406 and executable by the processor 1403 are the client application158, and potentially other applications. Also stored in the memory 1406may be a client device data store 1412 and other data. In addition, anoperating system may be stored in the memory 1406 and executable by theprocessor 1403.

With reference to FIG. 15, shown is a schematic block diagram of thecomputing environment 124 according to an embodiment of the presentdisclosure. The computing environment 124 includes one or more computingdevices 1500. Each computing device 1500 includes at least one processorcircuit, for example, having a processor 1503 and a memory 1506, both ofwhich are coupled to a local interface 1509. To this end, each computingdevice 1500 may comprise, for example, at least one server computer orlike device. The local interface 1509 may comprise, for example, a databus with an accompanying address/control bus or other bus structure ascan be appreciated. It is understood that there may be otherapplications that are stored in the memory 1506 and are executable bythe processor 1503 as can be appreciated.

Where any component discussed herein is implemented in the form ofsoftware, any one of a number of programming languages may be employedsuch as, for example, C, C++, C#, Objective C, Java®, JavaScript®, Perl,PHP, Visual Basic®, Python®, Ruby, Flash®, or other programminglanguages.

A number of software components are stored in the memory 1506 and areexecutable by the processor 1503. In this respect, the term “executable”means a program file that is in a form that can ultimately be run by theprocessor 1503. Examples of executable programs may be, for example, acompiled program that can be translated into machine code in a formatthat can be loaded into a random access portion of the memory 1506 andrun by the processor 1503, source code that may be expressed in properformat such as object code that is capable of being loaded into a randomaccess portion of the memory 1506 and executed by the processor 1503, orsource code that may be interpreted by another executable program togenerate instructions in a random access portion of the memory 1506 tobe executed by the processor 1503, etc. An executable program may bestored in any portion or component of the memory 1506 including, forexample, random access memory (RAM), read-only memory (ROM), hard drive,solid-state drive, USB flash drive, memory card, optical disc such ascompact disc (CD) or digital versatile disc (DVD), floppy disk, magnetictape, or other memory components.

The memory 1506 is defined herein as including both volatile andnonvolatile memory and data storage components. Volatile components arethose that do not retain data values upon loss of power. Nonvolatilecomponents are those that retain data upon a loss of power. Thus, thememory 1506 may comprise, for example, random access memory (RAM),read-only memory (ROM), hard disk drives, solid-state drives, USB flashdrives, memory cards accessed via a memory card reader, floppy disksaccessed via an associated floppy disk drive, optical discs accessed viaan optical disc drive, magnetic tapes accessed via an appropriate tapedrive, and/or other memory components, or a combination of any two ormore of these memory components. In addition, the RAM may comprise, forexample, static random access memory (SRAM), dynamic random accessmemory (DRAM), or magnetic random access memory (MRAM) and other suchdevices. The ROM may comprise, for example, a programmable read-onlymemory (PROM), an erasable programmable read-only memory (EPROM), anelectrically erasable programmable read-only memory (EEPROM), or otherlike memory device.

Also, the processor 1503 may represent multiple processors 1503 and/ormultiple processor cores and the memory 1506 may represent multiplememories 1506 that operate in parallel processing circuits,respectively. In such a case, the local interface 1509 may be anappropriate network that facilitates communication between any two ofthe multiple processors 1503, between any processor 1503 and any of thememories 1506, or between any two of the memories 1506, etc. The localinterface 1509 may comprise additional systems designed to coordinatethis communication, including, for example, performing load balancing.The processor 1503 may be of electrical or of some other availableconstruction.

Although the microcontroller application 1312, the client application158, the data processing application 139, and other various systemsdescribed herein may be embodied in software or code executed by generalpurpose hardware as discussed above, as an alternative the same may alsobe embodied in dedicated hardware or a combination of software/generalpurpose hardware and dedicated hardware. If embodied in dedicatedhardware, each can be implemented as a circuit or state machine thatemploys any one of or a combination of a number of technologies. Thesetechnologies may include, but are not limited to, discrete logiccircuits having logic gates for implementing various logic functionsupon an application of one or more data signals, application specificintegrated circuits (ASICs) having appropriate logic gates,field-programmable gate arrays (FPGAs), or other components, etc. Suchtechnologies are generally well known by those skilled in the art and,consequently, are not described in detail herein.

The flowcharts of FIGS. 8, 10, and 12 show the functionality andoperation of an implementation of portions of the microcontrollerapplication 1312, the client application 158, and the data processingapplication 139. If embodied in software, each block may represent amodule, segment, or portion of code that comprises program instructionsto implement the specified logical function(s). The program instructionsmay be embodied in the form of source code that comprises human-readablestatements written in a programming language or machine code thatcomprises numerical instructions recognizable by a suitable executionsystem such as a processor 1503 in a computer system or other system.The machine code may be converted from the source code, etc. If embodiedin hardware, each block may represent a circuit or a number ofinterconnected circuits to implement the specified logical function(s).

Although the flowcharts of FIGS. 8, 10, and 12 show a specific order ofexecution, it is understood that the order of execution may differ fromthat which is depicted. For example, the order of execution of two ormore blocks may be scrambled relative to the order shown. Also, two ormore blocks shown in succession in FIGS. 8, 10, and 12 may be executedconcurrently or with partial concurrence. Further, in some embodiments,one or more of the blocks shown in FIGS. 8, 10, and 12 may be skipped oromitted. In addition, any number of counters, state variables, warningsemaphores, or messages might be added to the logical flow describedherein, for purposes of enhanced utility, accounting, performancemeasurement, or providing troubleshooting aids, etc. It is understoodthat all such variations are within the scope of the present disclosure.

Also, any logic or application described herein, including themicrocontroller application 1312, the client application 158, the dataprocessing application 139, that comprises software or code can beembodied in any non-transitory computer-readable medium for use by or inconnection with an instruction execution system such as, for example, aprocessor 1503 in a computer system or other system. In this sense, thelogic may comprise, for example, statements including instructions anddeclarations that can be fetched from the computer-readable medium andexecuted by the instruction execution system. In the context of thepresent disclosure, a “computer-readable medium” can be any medium thatcan contain, store, or maintain the logic or application describedherein for use by or in connection with the instruction executionsystem.

The computer-readable medium can comprise any one of many physical mediasuch as, for example, magnetic, optical, or semiconductor media. Morespecific examples of a suitable computer-readable medium would include,but are not limited to, magnetic tapes, magnetic floppy diskettes,magnetic hard drives, memory cards, solid-state drives, USB flashdrives, or optical discs. Also, the computer-readable medium may be arandom access memory (RAM) including, for example, static random accessmemory (SRAM) and dynamic random access memory (DRAM), or magneticrandom access memory (MRAM). In addition, the computer-readable mediummay be a read-only memory (ROM), a programmable read-only memory (PROM),an erasable programmable read-only memory (EPROM), an electricallyerasable programmable read-only memory (EEPROM), or other type of memorydevice.

Further, any logic or application described herein, including themicrocontroller application 1312, the client application 158, the dataprocessing application 139, may be implemented and structured in avariety of ways. For example, one or more applications described may beimplemented as modules or components of a single application. Further,one or more applications described herein may be executed in shared orseparate computing devices or a combination thereof. For example, aplurality of the applications described herein may execute in the samecomputing device, or in multiple computing devices in the same computingenvironment 124. Additionally, it is understood that terms such as“application,” “service,” “system,” “engine,” “module,” and so on may beinterchangeable and are not intended to be limiting.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y, or at least one of Z to each be present.

It should be emphasized that the above-described embodiments of thepresent disclosure are merely possible examples of implementations setforth for a clear understanding of the principles of the disclosure.Many variations and modifications may be made to the above-describedembodiment(s) without departing substantially from the spirit andprinciples of the disclosure. All such modifications and variations areintended to be included herein within the scope of this disclosure andprotected by the following claims.

Therefore, the following is claimed:
 1. A system for telemedicine between a patient and a remotely-located medical professional, comprising: a plurality of sensor devices in data communication with a client device, wherein individual ones of the sensor devices comprise: a band configured to secure a respective one of the sensor devices to a portion of a body of a patient in accordance with a predefined arrangement of sensor devices; and processing circuitry configured to measure a position of the portion of the body of the patient to which the band is secured during a movement performed by the portion of the body of the patient; and the client device, wherein the client device comprises at least one hardware processor; and program instructions stored in memory and executable by the client device that, when executed, direct the client device to: establish a telemedicine session between the client device and a remote computing device operated by a medical professional located remotely from the patient; access sensor data measured by the plurality of sensor devices to determine the position of the portion of the body of the patient during the movement; render a graphical representation of the movement in a display accessible to the client device; render a graphical representation of an ideal body movement in the display; determine that the movement does not conform to the ideal body movement; render an indication that the movement does not conform to the ideal body movement in the display.
 2. The system of claim 1, wherein the predefined arrangement comprises a first one of the plurality of sensor devices being located at an upper torso of the body and a second one of the sensor devices being located at a lower torso of the body.
 3. The system of claim 1, wherein the predefined arrangement comprises a first one of the plurality of sensor devices being located at an upper portion of a leg of the body and a second one of the sensor devices being located at a lower portion of the leg of the body.
 4. The system of claim 1, wherein the program instructions that direct the client device to determine that the movement does not conform to the ideal body movement, when executed, further direct the client device to: determine that respective positions of individual ones of the plurality of sensor devices exceed a predefined threshold in relation to the ideal body movement.
 5. The system of claim 1, wherein the movement is a first movement the program instructions, when executed, further direct the client device to: access the sensor data measured by the plurality of sensor devices to determine a position of the portion of the body of the patient during a second movement; determine that the second movement conforms to the ideal body movement; and render an indication that the second movement conforms to the ideal body movement in the display.
 6. The system of claim 1, wherein the circuitry that measures the position of the portion of the body to which the band is secured further comprises an accelerometer and a gyroscope.
 7. The system of claim 1, wherein the circuitry that measures the position of the portion of the body of the patient to which the band is secured further comprises a magnetometer.
 8. The system of claim 1, wherein at least one of the plurality of sensor devices comprises a communication module configured to communicate the sensor data to the client device, the communication module comprising a BLUETOOTH® module, a BLUETOOTH® low-energy module, a ZIGBEE® module, or a near field communication (NFC) module.
 9. The system of claim 9, wherein the program instructions, when executed, further direct the client device to: identify a suggested change in the movement that conforms to the ideal body movement; and render an indication of the suggested change in the movement in the display.
 10. The system of claim 1, wherein the graphical representation of the ideal movement is rendered as an overlay on the graphical representation of the graphical representation of the movement in the display.
 11. A computer-implemented method for telemedicine between a patient and a remotely-located medical professional, comprising: accessing, by the at least one computing device, sensor data obtained by a plurality of sensor devices, wherein individual ones of the sensor devices comprise: a band configured to secure a respective one of the sensor devices to a portion of a body of the patient, in accordance with a predefined arrangement of sensor devices; and processing circuitry configured to measure a position of the portion of the body of the patient to which the band is secured during a movement performed by the portion of the body of the patient; establishing, by the at least one computing device, a telemedicine session between the at least one computing device and a remote computing device operated by a medical professional located remotely from the patient causing, by the at least one computing device, a graphical representation of the movement to be rendered in a display accessible to the at least one computing device; causing, by the at least one computing device, a graphical representation of an ideal body movement to be rendered in the display; determining, by the at least one computing device, that the movement does not conform to the ideal body movement; and causing, by the at least one computing device, an indication that the movement does not conform to the ideal body movement to be rendered in the display.
 12. The method of claim 11, wherein the predefined arrangement comprises a first one of the plurality of sensor devices being located at an upper torso of the body and a second one of the sensor devices being located at a lower torso of the body.
 13. The method of claim 11, wherein the predefined arrangement comprises a first one of the plurality of sensor devices being located at an upper portion of a leg of the body and a second one of the sensor devices being located at a lower portion of the leg of the body.
 14. The method of claim 11, wherein determining that the movement does not conform to the ideal body movement further comprises: determining, by the at least one computing device, that respective positions of individual ones of the plurality of sensor devices exceed a predefined threshold in relation to the ideal body movement.
 15. The method of claim 11, wherein the movement is a first movement, further comprising: accessing, by the at least one computing device, the sensor data measured by the plurality of sensor devices to determine a position of the portion of the body of the patient during a second movement; determining, by the at least one computing device, that the second movement conforms to the ideal body movement; and rendering, by the at least one computing device, an indication that the second movement conforms to the ideal body movement in the display.
 16. The method of claim 11, wherein the circuitry that measures the position of the portion of the body to which the band is secured further comprises an accelerometer and a gyroscope.
 17. The method of claim 11, wherein the circuitry that measures the position of the portion of the body of the patient to which the band is secured further comprises a magnetometer.
 18. The method of claim 11, wherein at least one of the plurality of sensor devices comprises a communication module configured to communicate the sensor data to the at least one computing device, the communication module comprising a BLUETOOTH® module, a BLUETOOTH® low-energy module, a ZIGBEE® module, or a near field communication (NFC) module.
 19. The method of claim 11, further comprising Identifying, by the at least one computing device, a suggested change in the movement that conforms to the ideal body movement; and render, by the at least one computing device, an indication of the suggested change in the movement in the display.
 20. The method of claim 11, wherein the graphical representation of the movement and the graphical representation of the ideal movement are rendered side-by-side in the display. 